Electrical mirroring by nand flash controller

ABSTRACT

Systems, apparatus and methods are provided for electrical mirroring implemented by a storage controller in a non-volatile storage system. In one embodiment, a non-volatile storage system may comprise a plurality of non-volatile storage devices and a storage controller. The storage controller may be configured to perform an electrical mirroring configuration process comprising: determining a system topology of the non-volatile storage system and which targets are in mirrored non-volatile storage devices and setting respective register bits in the storage controller for all targets in all mirrored non-volatile storage devices of the plurality of non-volatile storage devices.

TECHNICAL FIELD

This disclosure herein relates to the NAND flash controller architectureand implementation, particularly relates to the multi-drop multi-loadNAND interface topology where a number of NAND flash devices share acommon data bus with a NAND controller.

BACKGROUND

The evolution of the modern computing system is driven in-part by theemergence of the Solid State Drives (SSDs) that have demonstrated higherperformance of speed and latency over the traditional hard drives.Unlike hard drives that depend on the magnetism to store data, solidstate drives use NAND flash devices to achieve data storage. The NANDflash devices are a family of integrated circuits that are manufacturedby advanced process and assembly technologies to achieve multiple levelsof vertical stacking of storages units into a small footprint of packagefor high capacity of storage.

A typical SSD consists of a controller and a number of NAND flashdevices which are placed on and connected through a printer circuitboard (PCB) with a standard form factor for various consumer orenterprise usage models. The interface between the controller and NANDflash devices is grouped into channels, and a modern controller usuallyhas 4, 8 or 16 NAND channels. In order to achieve higher storagecapacity, the SSD needs to integrate more NAND flash devices into thePCB, thereby resulting in multiple NAND devices sharing a singlechannel. As a result, a multi-load or multi-drop PCB topology is oftenin place for the design of high-density SSD.

SUMMARY

A typical high capacity SSD multi-load PCB, whether in a M.2 or U.2 formfactor, often adopts a so-called clamshell topology of NAND flashplacement. In this topology, the NAND device on one side of a PCB may befully overlapped with the NAND device on the other side. Because thepins for a NAND device may be positioned on one side of the NAND device.One of the two NAND devices may be flipped for mounting and placement.The pin order for the flipped NAND device may be in a fully mirroredorder to that of the not-flipped NAND device. For example, in terms ofzero based indexing for an eight-bit data bus of DQ[0 . . . 7], thismeans that DQ[0] of the flipped NAND device will be aligned with DQ[7]of the not-flipped NAND device, DQ[1] of the flipped NAND device will bealigned with DQ[6] of the not-flipped NAND device, and so on and soforth.

This placement, however, introduces challenges to PCB layout and signalintegrity. For example, the same DQ pins of the flipped NAND device andthe not-flipped NAND device will be always in symmetrical positions withrespect to the center line of the NAND devices. That is, for example,DQ[0] of the flipped NAND device and DQ[0] of the not-flipped NANDdevice will be positioned on opposite sides at symmetrical positionswith respect to the center line of the NAND devices, DQ[1] of theflipped NAND device and DQ[1] of the not-flipped NAND device will bepositioned on opposite sides at symmetrical positions with respect tothe center line of the NAND devices, and so on and so forth. As aresult, if the two NAND devices in a mirrored configuration share onechannel, the same two DQ pins (e.g., DQ[0] of the flipped NAND deviceand DQ[0] of the not-flipped NAND device) are normally connected by viasplaced along the center line of the NAND devices, and there will be tworelatively long traces routed from the via to the pins. These tracelengths are usually in the range of 300-mil to 500-mil. On the otherhand, signal integrity specification for the high-speed NAND interfacerequires those trace lengths to be as short as possible, with maximumlength less than 200-mil. Having such long trace lengths tend to causeNAND performance degradation of one to two speed grades, which isparticularly a challenge for the high-capacity NAND device with heavycapacitive loading.

One approach to resolve the issue of signal integrity degradation may beto have two sets of the pin maps for the same NAND devices: a regularpin map and a reversed pin map. The reversed pin map may have a reverseddata pin order. That is, DQ[7] data pin in the reversed pin map may beput in the position for the DQ[0] data pin in the regular pin map, DQ[6]data pin in the reversed pin map may be put in the position for theDQ[1] data pin in the regular pin map, and so on and so forth.Therefore, a NAND device with a regular pin map may be matched with aNAND device with a reversed pin map and the data pins of the two NANDdevices may be matched and mirrored on two sides of a PCB. This willenable the via placement next to the pins and a very short trace about˜20-mil long to be routed from the via to the data pins. This solutionwill greatly enhance the PCB layout quality and resolve theaforementioned signal integrity degradation, to help enable the systemto reach much higher NAND interface speed. This technique of reversingor mirroring NAND Data pin order may be referred to as electricalmirroring.

The existing solution uses a dedicated electrical mirroring command sentfrom the controller to the NAND flash device right after deviceinitialization. The NAND device sees this command and then configuresitself into the mirrored mode in which the DQ pins are reversed from theregular pin map. That is, pin DQ0 is now configured as DQ7 inside theNAND device, DQ1 becomes DQ6, and so on and so forth. However, the NANDdevice has to be configured to support this electrical mirroringfeature.

Moreover, the electrical mirroring configuration command has zerotolerance of error. It has to be sent right after initialization, onceand for all. Because it is sent before any of the device trainings andtuning can be done, the chance of error and failure is significantlyhigh. Once it fails, the entire NAND device cannot be accessed until thenext power cycle because NAND devices do not have any RESET mechanism.And a power cycle is not desired by any of the client or enterprisesystem users. In fact, this electrical mirroring feature is oftenavoided because of the high failure rate.

Therefore, this is a need in the art for a more flexible and robustelectrical mirroring technique. A new electrical mirroring technique maybe developed and implemented within the NAND flash controller. Invarious embodiments, the NAND device that needs to be mirrored will haveits data pins connected in a reversed order to the controller's datapins. The controller may use firmware to determine whether electricalmirroring is enabled or disabled for each individual NAND device andimplement a set of registers with bits corresponding to each individualNAND device to be set by the firmware during a configuration process(e.g., power on initialization). Once the electrical mirroring isenabled for a NAND device, the controller may reverse the order of databits before sending it to the NAND device with reversed data pins, andalso reverse the order of data bits received from the NAND device withreversed data pins.

In an exemplary embodiment, there is provided a method that may comprisedetermining a system topology of a non-volatile storage system and whichtargets are in mirrored non-volatile storage devices, and settingrespective register bits in a storage controller for all targets in allmirrored non-volatile storage devices.

In another exemplary embodiment, there is provided a non-volatilestorage system. The non-volatile storage system may comprise a pluralityof non-volatile storage devices and a storage controller. The storagecontroller may be configured to perform an electrical mirroringconfiguration process comprising: determining a system topology of thenon-volatile storage system and which targets are in mirrorednon-volatile storage devices and setting respective register bits in thestorage controller for all targets in all mirrored non-volatile storagedevices of the plurality of non-volatile storage devices.

In yet another exemplary embodiment, there is provided a non-transitorymachine-readable medium. The non-transitory machine-readable medium mayhave computer instructions that when executed by a hardware processor,may cause the hardware processor to perform determining a systemtopology of a non-volatile storage system and which targets are inmirrored non-volatile storage devices and setting respective registerbits in a storage controller for all targets in all mirrorednon-volatile storage devices.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 schematically shows a non-volatile storage system in accordancewith an embodiment of the present disclosure.

FIG. 2 schematically shows a mirrored configuration of two non-volatilestorage devices in accordance with an embodiment of the presentdisclosure.

FIG. 3 schematically shows details of a mirrored configuration of twonon-volatile storage devices in accordance with an embodiment of thepresent disclosure.

FIG. 4 schematically shows a 2-channel non-volatile storage system inaccordance with an embodiment of the present disclosure.

FIG. 5 schematically shows read and write signaling for the 2-channelnon-volatile storage system in accordance with an embodiment of thepresent disclosure.

FIG. 6 is a flowchart of an electrical mirroring configuration processin accordance with an embodiment of the present disclosure.

FIG. 7 is a flowchart of a process for a write operation to a mirroredNAND device in accordance with an embodiment of the present disclosure.

FIG. 8 is a flowchart of a process for a read operation from a mirroredNAND device in accordance with an embodiment of the present disclosure.

DETAILED DESCRIPTION

Specific embodiments according to the present disclosure will now bedescribed in detail with reference to the accompanying figures. Likeelements in the various figures are denoted by like reference numeralsfor consistency.

The present disclosure provides systems and methods for electricalmirroring by a storage controller. In various embodiments, anon-volatile storage device in a non-volatile storage system such as anSSD may have its data pins connected in a reversed order to the storagecontroller's data pins. The controller may let firmware determinewhether electrical mirroring is enabled or disabled for each individualnon-volatile memory device and implement a set of registers with bitscorresponding to each individual non-volatile memory device to be set bythe firmware during a configuration process (e.g., power oninitialization). Once the electrical mirroring is enabled for anon-volatile memory device, the controller may reverse the order of databits before sending it to the non-volatile memory device with reverseddata pins, and also reverse the order of data bits received from thenon-volatile memory device with reversed data pins.

As used herein, a non-volatile memory device may be a computer storagedevice that can maintain stored information after being powered off, andthe stored information may be retrieved after being power cycled (turnedoff and back on). Non-volatile storage devices may include NAND flashmemories, NOR flash memories, magnetoresistive random Access Memory(MRAM), resistive random access memory (RRAM), phase change randomaccess memory (PCRAM), Nano-RAM, etc. In the description, a NAND flashmay be used as an example to demonstrate the controller implementedelectrical mirroring techniques. However, various embodiments accordingto the present disclosure may implement the techniques with other typesof non-volatile storage devices.

FIG. 1 schematically shows a non-volatile storage system 100 inaccordance with an embodiment of the present disclosure. Thenon-volatile storage system 100 may comprise a non-volatile storagecontroller 102 and a plurality of non-volatile memories (NVMs) 104. Thenon-volatile storage system 100 may provide data storage and/or accessto stored data to a host when it is coupled to the host. The pluralityof NVMs 104 may be, for example, NAND devices. In some embodiments, oneNAND device may comprise one NAND target that may be controlled by achip enable (CE) signal. In some other embodiments, one NAND device maycomprise a plurality of NAND targets (e.g., two targets) that each NANDtarget may have its own CE signal. As used herein, a storage target in aNVM storage device may be a unit of the storage device accessed by achip enable signal and may be simply referred to as a target. Theplurality of NVMs 104 may be connected to the non-volatile storagecontroller 102 by a plurality of channels with each target beingconnected to one channel and one channel shared by a plurality oftargets.

At least two NVMs of the plurality of NVMs 104 may be put in a mirroredconfiguration with one of the two NVMs at one side of a PCB and theother one of the two NVMs at the same position but opposite side thePCB. One of the two NVMs in the mirrored configuration may have datapins connected to the data bus of a channel in a regular order and theother one of the two NVMs in the mirrored configuration may have datapins connected to the data bus in a reversed order. As used herein, aNVM may be referred to as connected in a reversed order when its datapins are connected to a data bus in the reversed order and such areversely connected NVM may also be referred to as a flipped NVM, amirrored NVM, or a NVM in a mirrored mode. Further, a NVM connected in aregular data bit order may be referred to as a not-flipped NVM or anon-mirrored NVM.

The non-volatile storage controller 102 may comprise a processor 106, anon-transitory computer-readable storage media 108 and a plurality ofregisters 110. The processor 106 may be a computer processor, such as,but not limited to, a microprocessor or a microcontroller. In someembodiments, each of the plurality of registers 110 may have a pluralityof bits such that each target of the plurality of NVMs 104 may have acorresponding bit in the plurality of registers 110. The plurality ofregisters 110 may be configured to indicate which target of theplurality of NVMs 104 may be connected in a reversed order. In at leastone embodiment, some or all of the processor 106, the storage media 108and registers 110 may be integrated in one Integrated Circuit (IC) chip.

In one embodiment, each register of the plurality of registers 110 maybe assigned to one channel (e.g., one register per channel) and eachregister bit may be assigned to one target of one NVM device of thechannel (e.g., one bit per CE). In an example implementation, theregister may be referred to as DQ_MIRROR_EN as shown in Table 1 below,in which a bit may be set (e.g., set to one) to enable the electricalmirroring mode of a corresponding NAND target.

TABLE 1 Register definition of DQ_MIRROR_EN Bit- Register Name WidthDescription DQ_MIRROR_EN [15:0] Each bit corresponding to one CE 0 h:electrical mirror mode disabled 1 h: electrical mirror mode enabled(e.g., DQ0 coupled to DQ7 of data bus; DQ1 coupled to DQ6 of data bus; .. . ; DQ7 coupled to DQ0 of data bus)

In the example of Table 1, the register may have a 16-bit width. Witheach bit corresponding to one NAND target, the 16-bit register maysupport up to 16 targets per channel. The register bits may be set basedon system topology. Once a DQ_MIRROR_EN register bit is set to 1, thecontroller 102 may be notified that the corresponding NAND device (e.g.,the NAND device containing the corresponding target) has the data pinsconnected in a reverse order, so the controller may need to reverse thedata bit order internally before sending the data or after receiving thedata.

It should be noted that the example uses a register bit set to one (1)to indicate a corresponding NVM target may be connected in a reversedorder, but in some embodiments, a register bit set to zero (0) may beused to indicate a corresponding NVM target connected in a reversedorder. Moreover, it should be noted that the register may have adifferent width compared to the data bus. In the example shown above,the data bus may have a 8-bit width but the register may have 16 bits.

In some embodiments, the register bits of the plurality of registers 110corresponding to reversely connected NVMs of the plurality of NVMs 104may be set during an electrical mirroring configuration process. Forexample, the electrical mirroring configuration process may be part ofan initialization process of the non-volatile storage system 100performed when the non-volatile storage system 100 is powered on orreset. During the electrical mirroring configuration process, theprocessor 106 may execute computer instructions (e.g., firmware) storedin the non-transitory computer-readable storage media 108 to determinesystem topology and set the register bits based on the determined systemtopology. In some embodiments, the non-transitory computer-readablestorage media 108 may be a read only memory (e.g., Erasable ProgrammableRead Only Memory or other suitable ROM) typically used for storingfirmware, which may be low-level software instructions that whenexecuted by the processor 106, may cause the processor 106 to directlyaccess the registers of the controller, to configure, control andprogram the behaviors of the controller and the system.

In one embodiment, the processor 106 may execute computer instructionsstored in the non-transitory computer-readable storage media 108 todetermine which NVM of the plurality of NVMs may be connected in areversed order and set bits corresponding to targets in the reverselyconnected NVMs in the plurality of registers 110. For example, thefirmware may be programmed to designate one side as a regular orderconnection side (e.g., top side) and another side as the reversed orderconnection side (e.g., bottom side) and the processor 106 may determinewhich NVMs of the plurality of NVMs 104 may be positioned on the regularorder connection side and which NVMs of the plurality of NVMs 104 may bepositioned on the reversed order connection side.

During operations for the non-volatile storage system 100, thenon-volatile storage controller 102 may be configured to reverse databit orders before sending data to a reversely connected target and afterreceiving data from a reversely connected target. In one embodiment, theprocessor 106 may execute software instructions to check registers 110to determine whether a target is reversely connected and carry out datareversal operations if necessary. If a NAND target is connected inregular order, data may be sent from the non-volatile storage controller102 to the NAND target and received from the NAND target without anydata reversal operations.

If a NAND target is reversely connected (e.g., electrical mirroringregister bit enabled), for a write operation, the non-volatile storagecontroller 102 may reverse the data bit order from DQ[7:0] to DQ[0:7]before sending the data out on the data bus. That is, the non-volatilestorage controller 102 may swap data bits DQ[0] and DQ[7], swap databits DQ[1] and DQ[6], and so on for all data bits. The NAND device mayreceive the data in the original order of DQ[7:0]. That is, because thedata pins of the NAND device are connected to the data bus in a reversedorder, data pin DQ[0] of the NAND device may receive the data bit forDQ[0] in the non-volatile storage controller 102 before the reversaloperation, data pin DQ[1] of the NAND device may receive the data bitfor DQ[1] in the non-volatile storage controller 102 before the reversaloperation, and so on.

For a read operation with the mirrored NAND device, the non-volatilestorage controller 102 may receive the data of DQ[7:0] in the reversedorder as DQ[0:7]. The non-volatile storage controller 102 may determinethat the received data is in the reversed data bit order by checking thecorresponding register bit in the registers 110, and then perform a databit order reversal to recover the data in the original order as DQ[7:0].

FIG. 2 schematically shows two non-volatile storage devices 204 and 206in a mirrored configuration in accordance with an embodiment of thepresent disclosure. The two non-volatile storage devices 204 and 206 maybe attached to a same position but two opposite sides of a substrate orbase 202, respectively. That is, the two non-volatile storage devices204 and 206 may overlap with each other in a clamshell topology. Thesubstrate 202 may be a printed circuit board (PCB) or other suitablesubstrate. The two non-volatile storage devices 204 and 206 may be twoof the plurality of NVMs 104 and the substrate 202 may be a PCB in thenon-volatile storage system 100.

FIG. 3 schematically shows details of the two non-volatile storagedevices 204 and 206 in the mirrored configuration in accordance with anembodiment of the present disclosure. In the example of FIG. 3, the databus of a channel may be 8 bits. But in other embodiments, the data busof a channel may be 16 bits, 32 bits, or other suitable number of bits.The non-volatile storage device 204 may be positioned on a top side ofthe PCB 202 and have data pins DQ[0] 302, DQ[1] 304, DQ[2] 306, DQ[3]308, DQ[4] 310, DQ[5] 312, DQ[6] 314 and DQ[7] 316. The data pins DQ[0]302, DQ[1] 304, DQ[2] 306, DQ[3] 308, DQ[4] 310, DQ[5] 312, DQ[6] 314and DQ[7] 316 may be underneath the non-volatile storage device 204 andshown in dashed lines. The non-volatile storage device 206 may bepositioned on a bottom side of the PCB 202 and have data pins DQ[0] 318,DQ[1] 320, DQ[2] 324, DQ[3] 326, DQ[4] 328, DQ[5] 330, DQ[6] 332 andDQ[7] 334. The non-volatile storage device 206 and its data pins DQ[0]318, DQ[1] 320, DQ[2] 324, DQ[3] 326, DQ[4] 328, DQ[5] 330, DQ[6] 332and DQ[7] 334 may be also shown in dashed lines.

In the mirrored configuration of FIG. 3, DQ[0] 302 of the non-volatilestorage device 204 may be in a mirrored position of DQ[7] 334 of thenon-volatile storage device 206, DQ[1] 304 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[6] 332 of thenon-volatile storage device 206, DQ[2] 306 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[5] 330 of thenon-volatile storage device 206, DQ[3] 308 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[4] 328 of thenon-volatile storage device 206, DQ[4] 310 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[3] 326 of thenon-volatile storage device 206, DQ[5] 312 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[2] 324 of thenon-volatile storage device 206, DQ[6] 314 of the non-volatile storagedevice 204 may be in a mirrored position of DQ[1] 320 of thenon-volatile storage device 206 and DQ[7] 316 of the non-volatilestorage device 204 may be in a mirrored position of DQ[0] 318 of thenon-volatile storage device 206. As used herein, a mirrored position mayrefer to the same position but opposite side of the PCB.

As shown in FIG. 3, DQ[0] 302 of the non-volatile storage device 204 maybe connected to a via 336 by a trace 340 and DQ[7] 334 of thenon-volatile storage device 206 may be connected to the via 336 by atrace 342. The via 336 may be connected to a trace 348 for DQ[0] of thedata bus of a channel. DQ[7] 316 of the non-volatile storage device 204may be connected to a via 338 by a trace 344 and DQ[0] 318 of thenon-volatile storage device 206 may be connected to the via 338 by atrace 346. The via 338 may be connected to a trace 350 for DQ[7] of thedata bus of a channel. Because every pair of data pins connected by avia may be in mirrored positions, the traces connecting the data pins tothe via (e.g., traces 340, 342, 344 and 346) may be short. This mayallow for a good quality PCB layout and higher speed NAND interface. Thedata connections may be shown in dotted lines in FIG. 3 to indicate thatthey may be buried inside the PCB 202. It should be noted that otherdata connections (e.g., vias and traces for other data pins) may beomitted in FIG. 3 for simplicity.

FIG. 4 schematically shows a 2-channel non-volatile storage system 400in accordance with an embodiment of the present disclosure. The2-channel non-volatile storage system 400 may comprise a NAND controller402, a first NAND device 404.1 and a second NAND device 404.2. The firstNAND device 404.1 and the second NAND device 404.2 may be in a mirroredconfiguration. For example, the first NAND device 404.1 may be on a topside of a PCB and the second NAND device 404.2 may be on a bottom sideof the PCB. The 2-channel non-volatile storage system 400 may be anembodiment of the non-volatile storage system 100.

The first NAND device 404.1 may comprise two NAND targets: NAND target406.1 and NAND target 406.2. The second NAND device 404.2 may alsocomprise two NAND targets: NAND target 406.3 and NAND target 406.4. TheNAND target 406.1 of the first NAND device 404.1 and the NAND target406.3 of the second NAND device 404.2 may be connected to a firstchannel 410.1. The NAND target 406.2 of the first NAND device 404.1 andthe NAND target 406.4 of the second NAND device 404.2 may be connectedto a second channel 410.2. The NAND target 406.1 may be selected fordata operations by a CE signal on a CE signal line 408.1. The NANDtarget 406.2 may be selected for data operations by a CE signal on a CEsignal line 408.2. The NAND target 406.3 may be selected for dataoperations by a CE signal on a CE signal line 408.3. The NAND target406.4 may be selected for data operations by a CE signal on a CE signalline 408.4. In one embodiment, the NAND device 404.2 may be a mirroreddevice. That is, the data pins of the NAND device 404.2 may be connectedto the data bus for the channels 410.1 and 410.2 in a reversed order.

It should be noted that each channel may have its own data bus. In someembodiments, a NAND storage device may have two sets of data pins fortwo channels that may be independent from each other. In the example inFIG. 4, a NAND storage device may have two targets so each target in onedevice may be coupled to one channel. In some other embodiments, a NANDstorage device may have more than two targets and some targets may shareone set of data pins for a data bus. For example, if a device has 4targets, then each 2 targets may share one set of data pins for a databus of a channel. If the device has 8 targets, then each 4 targets mayshare a set of data pins for a data bus of a channel. But regardless howmany targets are in a NAND storage device, if a NAND storage device isin a side of a PCB that data pins of the NAND storage device need to beconnected in a reversed order, then all targets in the NAND storagedevice may have corresponding register bits set in the controller toindicate that they are reversely connected.

In one embodiment, the NAND controller 402 may comprise two electricalmirroring registers: one for channel 410.1 and another for channel410.2. In the electrical mirroring register for channel 410.1, a bitcorresponding to the NAND target 406.3 may be set during an electricalmirroring configuration process. Also, in the electrical mirroringregister for channel 410.2, a bit corresponding to the NAND target 406.4may be set during an electrical mirroring configuration process. Forwrite and read operations, the NAND controller 402 may check theelectrical mirroring registers to determine whether a NAND target isreversely connected and whether the data bit order reversal operationmay be necessary.

FIG. 5 schematically shows read and write signaling for the 2-channelnon-volatile storage system 400 of FIG. 4 in accordance with anembodiment of the present disclosure. Signal line 502 may represent thecycle types. The signal block 516 may represent a write operation cycleto a NAND target of regular data bit order (e.g., a CE signal on the CEsignal lines 408.1 or 408.2). The signal block 518 may represent a writeoperation cycle to a NAND target of reversed data bit order (e.g., a CEsignal on the CE signal lines 408.3 or 408.4). The signal block 520 mayrepresent a read operation cycle from a NAND target of regular data bitorder (e.g., a CE signal on the CE signal lines 408.1 or 408.2). Thesignal block 522 may represent a read operation cycle from a NAND targetof reversed data bit order (e.g., a CE signal on the CE signal lines408.3 or 408.4).

Signal line 504 may represent data in the NAND controller 402. In blocks524, 526, 528 and 530, data in the NAND controller 402 are all inregular bit order (e.g., DQ[7:0]).

Signal line 506 may represent data on the data bus. In block 532, dataon the data bus may be in the regular data bit order because the cycletype is a write operation cycle for a regular data bit order NANDtarget. In block 534, data on the data bus may be in the reversed databit order (e.g., DQ[0:7]) because the cycle type is a write operationcycle for a NAND target in a mirrored NAND device. In block 536, data onthe data bus may be in the regular data bit order because the cycle typeis a read operation cycle for a regular data bit order NAND target. Inblock 538, data on the data bus may be in the reversed data bit orderbecause the cycle type is a read operation cycle for a NAND target in amirrored NAND device.

Signal line 508 may represent data on the NAND device 404.1 and signalline 510 may represent data on the NAND device 404.2. In blocks 540 and542, data in the NAND device 404.1 may be in regular data bit order.Because the NAND controller 402 may reverse data bit orders beforesending data to the mirrored NAND device and after receiving data fromthe mirrored NAND device, in blocks of 544 and 546, data in the NANDdevice 404.2 may also be in regular data bit order. Signal line 512 mayrepresent a CE signal line for the NAND device 404.1 (e.g., CE signalline 408.1 or CE signal line 408.2) and show that the CE signal may beset (e.g., pulled down) in signals 548 (e.g., write to a regular databit order NAND target) and 550 (e.g., read from a regular data bit orderNAND target). Signal line 514 may represent a CE signal line for theNAND device 404.2 (e.g., CE signal line 408.3 or CE signal line 408.4)and show that the CE signal may be set (e.g., pulled down) in signals552 (e.g., write to a NAND target in a mirrored NAND device) and 554(e.g., read from a NAND target in a mirrored NAND device).

As shown in FIG. 5, during a write operation to the NAND device 404.2(e.g., a CE signal set on either signal line 408.3 or signal line408.4), the NAND controller 402 may reverse the data bit order fromregular data bit order (e.g., DQ[7:0]) to the reversed data bit order(e.g., DQ[0:7]) before sending the data on the data bus. Thus, the NANDdevice 404.2 may receive the data in the regular data bit order becausethe order is reversed again though the data pins to data bus connection.

To conduct a read operation from the NAND device 404.2 (e.g., a CEsignal set on either signal line 408.3 or signal line 408.4), thecontroller 402 may receive the data on the data bus in a reversed order(e.g., DQ[0:7]). By determining that the electrical mirroring isenabled, the controller 402 may perform another data bit order reversalto recover data in the original data bit order DQ[7:0].

In both read and write operations for a mirrored device (e.g., the NANDdevice 404.2), data on the NAND channel data bus may always have a databit order reversed from the data bit order of the data inside the NANDcontroller 402. The NAND controller 402 may manage the data bit order byperforming a round of data reversal for the electrically mirroreddevices. In contrast, the NAND device 404.1 may be connected to the databus in a matching order as DQ[7:0] to DQ[7:0], and the electricalmirroring is not enabled for the NAND device 404.1. So for the NANDdevice 404.1, data on the data bus may be always in the same order asthe data inside the controller 402.

FIG. 6 is a flowchart for an electrical mirroring configuration process600 in accordance with an embodiment of the present disclosure. In block602, a system topology of a non-volatile storage system and whichtargets are in mirrored non-volatile storage devices may be determined.For example, in some embodiments, the NAND controller 102 may beconfigured to determine the system topology and which targets are inmirrored non-volatile storage devices for the non-volatile storagesystem 100 during an electrical mirroring configuration process. In oneembodiment, the system topology may include how many non-volatilestorage devices are in the non-volatile storage system 100, the NVMs 104are connected to the controller 102 in how many channels, how manytargets are in each of the NVMs 104 and which targets in the NVMs 104belong to which channel.

In block 604, respective register bits in a storage controller may beset for all targets in all mirrored non-volatile storage devices. Forexample, in some embodiments, the NAND controller 102 may comprise aplurality of registers. In one embodiment, one channel may be assignedone register with each bit of the register corresponding to one targetin the channel. Register bits corresponding to targets in mirrorednon-volatile storage devices may be set (e.g., electrical mirroringenabled).

FIG. 7 is a flowchart of a process 700 for a write operation to amirrored NAND device in accordance with an embodiment of the presentdisclosure. In block 702, a non-volatile storage device of thenon-volatile storage system may be determined to be reversely connectedbased on a corresponding register bit being set. In one embodiment, aregister bit corresponding to a target in a mirrored NVM may be set toindicate that the data pins for the target may be reversely connected.In block 704, a data bit order for data to be stored in the non-volatilestorage device may be reversed at the storage controller. The data to bestored in the non-volatile storage device may be in an original data bitorder at the storage controller before the reversal. In block 706, thedata may be sent in a reversed data bit order on a data bus for the datato be stored in the non-volatile storage device. In various embodiments,because the target is in a mirrored storage device, the data pins arereversely connected, the data will be received and stored in themirrored storage device in its original data bit order.

FIG. 8 is a flowchart of a process 800 for a read operation from amirrored NAND device in accordance with an embodiment of the presentdisclosure. In block 802, data may be received at the storage controllerfrom a non-volatile storage device of the non-volatile storage system.In block 804, the storage controller may determine that the non-volatilestorage device is a mirrored non-volatile storage device based on acorresponding register bit being set. In block 806, the storagecontroller may reverse a data bit order for the received data.

Each of the processes 600, 700 and 800 may be implemented using software(e.g., executable by a computer processor (CPU, GPU, or both)), hardware(e.g., a field-programmable gate array (FPGA) or an application-specificIC (ASIC), firmware, or any suitable combination of the three. In oneembodiment, for example, the electrical mirroring configuration process600 may be implemented in hardware circuity (e.g., FPGA or ASIC). Inanother embodiment, the electrical mirroring configuration process 600may be implemented in firmware and stored in a read-only memory (e.g.,storage 108), and performed by the processor 106 executing the firmware.Moreover, the processes 700 and 800 may be programmed in computerprocessor executable instructions, stored in a non-transitorymachine-readable medium (e.g., NVMs 104, CD, DVD, etc.) and performed bya computer processor (e.g., a microprocessor or a microcontroller)executing the executable instructions.

In an exemplary embodiment, there is provided a method that may comprisedetermining a system topology of a non-volatile storage system and whichtargets are in mirrored non-volatile storage devices, and settingrespective register bits in a storage controller for all targets in allmirrored non-volatile storage devices.

In one embodiment, the system topology may include how many non-volatilestorage devices are in the non-volatile storage system, the non-volatilestorage devices are connected to a storage controller in how manychannels, how many targets are in each of the non-volatile storagedevices and which targets in the non-volatile storage devices belong towhich channel.

In one embodiment, the storage controller may comprise a plurality ofregisters with each register corresponding to one channel.

In one embodiment, the method may further comprise determining that anon-volatile storage device of the non-volatile storage system is amirrored non-volatile storage device based on a corresponding registerbit being set, reversing a data bit order at the storage controller fordata in an original data bit order to be stored in the non-volatilestorage device and sending the data in a reversed data bit order on adata bus connected to the non-volatile storage device for the data to bestored in the original data bit order in the non-volatile storagedevice.

In one embodiment, the method may further comprise receiving data at thestorage controller from a non-volatile storage device of thenon-volatile storage system, determining that the non-volatile storagedevice is a mirrored non-volatile storage device based on acorresponding register bit being set and reversing a data bit order atthe storage controller for the received data.

In one embodiment, each mirrored non-volatile storage device is pairedwith a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of the mirrored non-volatile storagedevice may be connected by vias to data pins of the non-mirrorednon-volatile storage device in a reversed order.

In another exemplary embodiment, there is provided a non-volatilestorage system. The non-volatile storage system may comprise a pluralityof non-volatile storage devices and a storage controller. The storagecontroller may be configured to perform an electrical mirroringconfiguration process comprising: determining a system topology of thenon-volatile storage system and which targets are in mirrorednon-volatile storage devices and setting respective register bits in thestorage controller for all targets in all mirrored non-volatile storagedevices of the plurality of non-volatile storage devices.

In one embodiment, the system topology may include how many non-volatilestorage devices are in the non-volatile storage system, the non-volatilestorage devices are connected to a storage controller in how manychannels, which of the non-volatile storage devices are mirrorednon-volatile storage devices and which targets in the non-volatilestorage devices belong to which channel.

In one embodiment, the storage controller may comprise a plurality ofregisters with each register corresponding to one channel.

In one embodiment, the storage controller may be further configured todetermine that a non-volatile storage device of the plurality ofnon-volatile storage devices is a mirrored non-volatile storage devicebased on a corresponding register bit being set, reverse a data bitorder at the storage controller for data in an original data bit orderto be stored in the non-volatile storage device and send the data in areversed data bit order on a data bus connected to the non-volatilestorage device for the data to be stored in the original data bit orderin the non-volatile storage device.

In one embodiment, the storage controller may be further configured toreceive data at the storage controller from a non-volatile storagedevice of the plurality of non-volatile storage devices, determine thatthe non-volatile storage device is a mirrored non-volatile storagedevice based on a corresponding register bit being set and reverse adata bit order at the storage controller for the received data.

In one embodiment, each mirrored non-volatile storage device may bepaired with a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of the mirrored non-volatile storagedevice may be connected by vias to data pins of the non-mirrorednon-volatile storage device in a reversed order.

In one embodiment, the storage controller may comprise a non-transitorycomputer-readable storage media storing firmware and a processorconfigured to execute the firmware to perform the electrical mirroringconfiguration process.

In one embodiment, the storage controller may comprise hardwarecircuitry configured to perform the electrical mirroring configurationprocess.

In yet another exemplary embodiment, there is provided a non-transitorymachine-readable medium. The non-transitory machine-readable medium mayhave computer instructions that when executed by a hardware processor,may cause the hardware processor to perform determining a systemtopology of a non-volatile storage system and which targets are inmirrored non-volatile storage devices and setting respective registerbits in a storage controller for all targets in all mirrorednon-volatile storage devices.

In one embodiment, the system topology may include how many non-volatilestorage devices are in the non-volatile storage system, the non-volatilestorage devices are connected to a storage controller in how manychannels, how many targets are in each of the non-volatile storagedevices and which targets in the non-volatile storage devices belong towhich channel.

In one embodiment, the storage controller may comprise a plurality ofregisters with each register corresponding to one channel.

In one embodiment, the computer instructions, when executed by thehardware processor, may further cause the hardware processor to perform:determining that a non-volatile storage device of the non-volatilestorage system is a mirrored non-volatile storage device based on acorresponding register bit being set, reversing a data bit order at thestorage controller for data in an original data bit order to be storedin the non-volatile storage device and sending the data in a reverseddata bit order on a data bus connected to the non-volatile storagedevice for the data to be stored in the original data bit order in thenon-volatile storage device.

In one embodiment, the computer instructions, when executed by thehardware processor, may further cause the hardware processor to perform:receiving data at the storage controller from a non-volatile storagedevice of the non-volatile storage system, determining that thenon-volatile storage device is a mirrored non-volatile storage devicebased on a corresponding register bit being set and reversing a data bitorder at the storage controller for the received data.

In one embodiment, each mirrored non-volatile storage device may bepaired with a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of the mirrored non-volatile storagedevice may be connected by vias to data pins of the non-mirrorednon-volatile storage device in a reversed order.

Any of the disclosed methods and operations may be implemented ascomputer-executable instructions (e.g., software code for the operationsdescribed herein) stored on one or more computer-readable storage media(e.g., non-transitory computer-readable media, such as one or moreoptical media discs, volatile memory components (such as DRAM or SRAM),or nonvolatile memory components (such as hard drives)) and executed ona device controller (e.g., firmware executed by ASIC). Any of thecomputer-executable instructions for implementing the disclosedtechniques as well as any data created and used during implementation ofthe disclosed embodiments can be stored on one or more computer-readablemedia (e.g., non-transitory computer-readable media).

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

What is claimed is:
 1. A method, comprising: enabling electricalmirroring for a non-volatile storage device by setting a correspondingregister bit for the non-volatile storage device in a storagecontroller; receiving data to be stored in the non-volatile storagedevice, the data being received in an original data bit order; reversinga data bit order at the storage controller for the received data togenerate data in a reversed data bit order; and sending the data in thereversed data bit order on a data bus connected to the non-volatilestorage device for the received data to be stored in the original databit order in the non-volatile storage device.
 2. The method of claim 1,wherein the non-volatile storage device is a mirrored non-volatilestorage device that has its data pins coupled to the data bus in areversed pin order.
 3. The method of claim 1, wherein the non-volatilestorage device is a part of a non-volatile storage system and the methodfurther comprises: determining a system topology of the non-volatilestorage system including which targets are in mirrored non-volatilestorage devices, wherein a target is a storage unit controlled by a chipenable signal, all mirrored non-volatile storage devices have theirrespective data pins coupled to the data bus in the reversed order, andeach non-volatile storage device of the non-volatile storage systemcomprises one or more targets; and setting respective register bits inthe storage controller for all targets in all mirrored non-volatilestorage devices.
 4. The method of claim 3, wherein the system topologyfurther includes how many non-volatile storage devices are in thenon-volatile storage system, the non-volatile storage devices areconnected to the storage controller in how many channels, how manytargets are in each of the non-volatile storage devices and whichtargets in the non-volatile storage devices belong to which channel. 5.The method of claim 4, wherein each mirrored non-volatile storage deviceis paired with a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of each mirrored non-volatile storagedevice are connected by vias to data pins of a respectively pairednon-mirrored non-volatile storage device in a reversed order.
 6. Themethod of claim 4, wherein the storage controller comprises a pluralityof registers with each register corresponding to one channel, and eachregister of the plurality of registers has a plurality of register bitswith each register bit corresponding to one target.
 7. The method ofclaim 1, further comprising: receiving stored data at the storagecontroller read from the non-volatile storage device; determining thatthe non-volatile storage device is a mirrored non-volatile storagedevice based on the corresponding register bit being set; and reversinga data bit order at the storage controller for the stored data.
 8. Anon-volatile storage system, comprising: a plurality of non-volatilestorage devices; and a storage controller configured to: enableelectrical mirroring for a non-volatile storage device of the pluralityof non-volatile storage devices by setting a corresponding register bitfor the non-volatile storage device in the storage controller; receivedata to be stored in the non-volatile storage device, the data beingreceived in an original data bit order; reverse a data bit order at thestorage controller for the received data to generate data in a reverseddata bit order; and send the data in the reversed data bit order on adata bus connected to the non-volatile storage device for the receiveddata to be stored in the original data bit order in the non-volatilestorage device.
 9. The non-volatile storage system of claim 8, whereinthe non-volatile storage device is a mirrored non-volatile storagedevice that has its data pins coupled to the data bus in a reversed pinorder.
 10. The non-volatile storage system of claim 8, wherein thestorage controller is further configured to: determine a system topologyof the non-volatile storage system including which targets are inmirrored non-volatile storage devices, wherein a target is a storageunit controlled by a chip enable signal, all mirrored non-volatilestorage devices have their respective data pins coupled to the data busin the reversed order, and each non-volatile storage device of thenon-volatile storage system comprises one or more targets; and setrespective register bits in the storage controller for all targets inall mirrored non-volatile storage devices.
 11. The non-volatile storagesystem of claim 10, wherein the system topology further includes howmany non-volatile storage devices are in the non-volatile storagesystem, the non-volatile storage devices are connected to the storagecontroller in how many channels, how many targets are in each of thenon-volatile storage devices and which targets in the non-volatilestorage devices belong to which channel.
 12. The non-volatile storagesystem of claim 11, wherein each mirrored non-volatile storage device ispaired with a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of each mirrored non-volatile storagedevice are connected by vias to data pins of a respectively pairednon-mirrored non-volatile storage device in a reversed order.
 13. Thenon-volatile storage system of claim 11, wherein the storage controllercomprises a plurality of registers with each register corresponding toone channel, and each register of the plurality of registers has aplurality of register bits with each register bit corresponding to onetarget.
 14. The non-volatile storage system of claim 8, wherein thestorage controller is further configured to: receive stored data at thestorage controller read from the non-volatile storage device; determinethat the non-volatile storage device is a mirrored non-volatile storagedevice based on the corresponding register bit being set; and reverse adata bit order at the storage controller for the stored data.
 15. Anon-transitory machine-readable medium having computer instructions,wherein the computer instructions, when executed by a hardwareprocessor, cause the hardware processor to perform: enable electricalmirroring for a non-volatile storage device by setting a correspondingregister bit for the non-volatile storage device in a storagecontroller; receive data to be stored in the non-volatile storagedevice, the data being received in an original data bit order; reverse adata bit order at the storage controller for the received data togenerate data in a reversed data bit order; and send the data in thereversed data bit order on a data bus connected to the non-volatilestorage device for the received data to be stored in the original databit order in the non-volatile storage device.
 16. The non-transitorymachine-readable medium of claim 15, wherein the non-volatile storagedevice is a mirrored non-volatile storage device that has its data pinscoupled to the data bus in a reversed pin order.
 17. The non-transitorymachine-readable medium of claim 15, wherein the non-volatile storagedevice is a part of a non-volatile storage system, and the computerinstructions, when executed by the hardware processor, further cause thehardware processor to perform: determine a system topology of thenon-volatile storage system including which targets are in mirrorednon-volatile storage devices, wherein a target is a storage unitcontrolled by a chip enable signal, all mirrored non-volatile storagedevices have their respective data pins coupled to the data bus in thereversed order, and each non-volatile storage device of the non-volatilestorage system comprises one or more targets; and set respectiveregister bits in the storage controller for all targets in all mirrorednon-volatile storage devices.
 18. The non-transitory machine-readablemedium of claim 17, wherein the system topology further includes howmany non-volatile storage devices are in the non-volatile storagesystem, the non-volatile storage devices are connected to the storagecontroller in how many channels, how many targets are in each of thenon-volatile storage devices and which targets in the non-volatilestorage devices belong to which channel, and wherein the storagecontroller comprises a plurality of registers with each registercorresponding to one channel, and each register of the plurality ofregisters has a plurality of register bits with each register bitcorresponding to one target.
 19. The non-transitory machine-readablemedium of claim 18, wherein each mirrored non-volatile storage device ispaired with a non-mirrored non-volatile storage device in a mirroredconfiguration in which data pins of each mirrored non-volatile storagedevice are connected by vias to data pins of a respectively pairednon-mirrored non-volatile storage device in a reversed order.
 20. Thenon-transitory machine-readable medium of claim 15, wherein the computerinstructions, when executed by the hardware processor, further cause thehardware processor to perform: receive stored data at the storagecontroller read from the non-volatile storage device; determine that thenon-volatile storage device is a mirrored non-volatile storage devicebased on the corresponding register bit being set; and reverse a databit order at the storage controller for the stored data.